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ABSTRACT 

The prospective use of low fidelity simulation and gaming in aviation training is high, and may facilitate individual, 
personal training needs in usually asynchronous training setting. Without direct feedback from, or intervention by, an 
instructor, adaptivity of the training environment is in high demand to ensure training sessions maintain an optimal 
training value to the trainee. In game design theory, the flow principle is used to provide an optimally engaging 
experience, whereas its equivalent in instructional design theory is maintaining the optimal cognitive load by adjusting 
the task complexity or by scaffolding. The control of these principles can be based on user activity or performance. 
Alternatively, brain measures may be used to control the learning experience of professionals. This paper explores the 
options for using brain measures for professional gaming and provides results of a pilot study. Based on the pilot study, it 
is concluded that brain measures may be a viable but demanding mechanism for optimizing the learning process. 
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1. INTRODUCTION 

Aviation has a long history of using simulation for training purposes. In particular the yearly recurrent and 
compulsory training for pilots is provided in mostly high-end simulators. There is growing insight that the 
standard curriculum is in need of revision. A wider use of training media may be required to ensure more 
training goals are covered in a better way while addressing more personal needs. PC based simulation and 
game based learning are considered candidates that partly replace and partly extend training on high end 
simulators. As training time is very expensive and personal needs require flexible solutions, a considerable 
part of new training options will require unscheduled and mobile activity, in which the pilot is more in 
control of what, how and when to train. This may lead to new organizational and regulatory mechanisms to 
register and accredit training as well as new instructional techniques to support the personal training needs 
without immediate availability of an instructor. Without direct feedback from, or intervention by, an 
instructor, adaptivity of the training environment is in high demand to ensure training sessions maintain an 
optimal value to the trainee. 

In game design theory, the flow principle is used to provide an optimally engaging experience, whereas 
its equivalent in instructional design theory is maintaining the optimal cognitive load (Paas, Tuovinen, 
Tabbers, & Van Gerven, 2003). Good game experience requires the player to be in a ‘flow’ state of mind 
(Csikszentmihalyi, 1991) which is feeling competent but challenged while being immersed in the game. This 
requires game designers to build up game events and levels that are neither too easy (boring) nor too difficult 
(frustrating) while ensuring that challenging periods are balanced with more relaxing periods without losing 
the players’ attention. Instructional design sequencing principles have similar goals which are achieved by 
increasing the task difficulty and by scaffolding principles (supporting or automating part of the tasks, such 
as the trainer wheels for learning to ride a bicycle), see Van Merrienboer & Kirschner, 2007. Adaptive 
training regulated by combining measures of performance and mental effort has shown to accelerate the 
learning curve for Air Traffic Control (Salden, Paas, Broers, & Van Merrienboer, 2004) and Flight 
Management System training (Salden, Paas, van der Pal, & Van Merrienboer, 2004). The learner’s cognitive 
load needs to be in an optimal band to ensure efficient learning. Over- or understimulation leads to frustration 
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or boredom and results in inefficient or even ineffective training. The principles of flow and the optimization 
of cognitive load both aim to control a learning curve, although they differ in which technique is applied. In 
gaming, the focus is centered on experience, whereas the focus in training is on performance. 

In training, cognitive load is usually controlled by measuring the load and performance after completion 
of a learning task. For aviation training to be fully effective, the events need to adapt to the cognitive load of 
events within a training session. Performance measured during the simulation or game is labeled as in-game 
measurement or stealth assessment (Shute, Ventura, Bauer, & Zapata-Rivera, 2009), techniques that require a 
coherent assessment framework, a user model and considerable further research and development (Baalsrud 
Hauge et. al. 2015) before well-grounded and practical use for automated adaptive training is achieved. Real 
time measures of mental states that reflect the experience of cognitive load (such as attention, engagement, 
situation awareness and boredom) should be part of such a framework. In this paper, brain measures are 
explored as a potential instrument for controlling the learning activity by automatically adjusting events in 
the learning scenario to ensure an optimal learning experience. Brain measures for attention and cognitive 
load are of particular interest. 


2. USING BCI FOR ADAPTIVE TRAINING 

Brain Computer Interface (BCI) stands for a range of techniques that support the brain to control a device 
without using muscles. For adaptive training, the trainee does not control the training tasks, events, setting or 
feedback by active thought or sheer will power (which is known as active BCI), but in a more indirect and 
involuntary way (also known as passive BCI), based on the measured amount of attention or relaxation, 
engagement or drowsiness, or other relevant mental states. BCI techniques for real, daily training will require 
a non-invasive, easy to use device. Wireless EEG devices with dry sensors may be candidates for practical 
BCI. There are several commercial EEG devices on the market that seem to be suitable. A range of validation 
studies have revealed some application areas as well as the limitations of these ‘simple’ devices. 

EEG (electroencephalography) is a well-known technique to measure the electric activity of the brain 
(groups of neurons firing simultaneously) on the scalp. After removal of e.g., muscle generated artifacts, 
EEG contains oscillations of various frequencies (from 0.1-100Hz) and amplitudes (up to 200 microvolts). 
These vary as a result of processing sensory input and internal mental activity. As a result, EEG is different 
on the various parts of the brain, although precise location is not a strong feature of EEG. The different 
frequencies have been found to indicate certain mental states and emotions. For example, a low amplitude in 
the region of 8-12 Hz indicates attention, especially in combination with a high amplitude in the 13-30 Hz 
range. The frequency bands have been labeled by Greek letters (alpha to gamma), and include sub-bands like 
low and high beta. The (sub-)bands and certain composite measures indicate a variety of mental states and 
functions. An example of a composite measure is the Task Engagement Index, calculated by beta / (alpha + 
theta), which has been constructed for adaptive automated flight control (Pope et al , 1995) and has for 
instance been applied in measuring immersion during game play (McMahan, Parberry, & Parsons, 2015). 

Raw EEG data is normally recorded for later analysis, which requires powerful computers, complex 
algorithms and time. BCI cannot work this way, as specific EEG frequencies or indexes need to be calculated 
and corrected for muscle activity in real time. This requires a highly dedicated algorithm tuned to the specific 
sensors and locations, hardwired into a small chip in the device itself. This in turn demands considerable 
research and development, and the companies consequently consider the results as proprietary, including 
basic information on the frequency bands or composite measures used. A number of currently available BCI 
measures are presented in Table 1. 

Table 1. Preprocessed EEG measures in commercial EEG devices 


Neurosky Mindwave Emotive Insight 


Meditation 

Relaxation 


Interest/ Affinity 

Attention 

Focus 


Engagement 


Instantaneous excitement 


Long term excitement 
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For the concepts of flow and optimal cognitive load, EEG indicators for cognitive load/task difficulty, 
attention, and task engagement are relevant. Task difficulty is associated with theta and alpha oscillations. 
Theta (which is most prominent in the frontal midline) is increased in high difficulty tasks in flight simulators 
(Smith & Gevins 2005), Alpha indicates the cognitive load of visual/auditory tasks (Gerlic 1999). For 
military pilots, alpha is found to decrease during demanding air refueling and landing exercises (Sterman et 
al, 1994). There are indications that the high alpha band is more related to (verbal) long term memory 
activities and theta to working memory (Antonenko, Paas, Grubner, & van Gog, 2010). Theta is therefore a 
candidate trigger to control overstimulation and high alpha is a candidate trigger to control understimulation 
in a scenario. 


Timed interval (frequent) 



Figure 1. Expected progress of attention level based on different methods for triggering simulator events; timed interval 
(frequent as well as infrequent intervals) and BCI controlled (set to target attention optimal values) 


3. METHOD 

To determine if BCI devices can be used to more effectively trigger scenario events, a pilot experiment is set 
up. The between subjects design compares two conditions - time interval triggered vs mental state triggered 
simulator events. Participants are asked to perform a short training by flying a helicopter around an urban 
area in a low fidelity (gaming) simulated environment. The objective of the training is to familiarise with 
basic helicopter control mechanisms (pitch, role and yaw). The training task consists of flying through 
consecutive augmented cues, a kind of ‘virtual checkpoints’ in the sky. These hoop-shaped checkpoints are 
placed in a track configuration, and are located on different heights. 

Eight participants (7 male, 1 female; age ranges from 21 to 36 years with an average of 27) are randomly 
assigned to either one condition. The test starts with a 1 minute familiarisation of the task, where the task is 
explained. Once the participants understand the task, the helicopter control training commences. 

The training task is identical for all participants: to learn to control the helicopter by flying through a set 
of consecutive digital ‘checkpoints’. This training takes # minutes to complete. Depending on the assigned 
condition, the task either automatically increases in difficulty (time based interval condition, ‘A’) or varies in 
difficulty depending on the participant’s attention level (mental state based condition, ‘B’). 

In the time based interval condition (A), to increase difficulty the checkpoint diameter decreases 
gradually over 5 minutes. This reduction triggers regardless of how well the trainee performs. In the mental 
state based (B) condition, the task complexity changes on the basis of the level of attention of the trainee. 
When strained by the task, trainee attention will increase, thus increasing the diameter of the checkpoints. 
When the task no longer requires high attention (through increased mastery of the controls), the checkpoint 
diameter will remain constant. When the attention level becomes too low, the checkpoint diameter will 
dynamically decrease, thus increasing the task complexity. During training, the checkpoint diameter 
decreases when the participant’s attention level is higher than 70, and increases when attention level is lower 
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than 30 on a scale from 0 to 100. An optimal level of attention is achieved between 30 and 70. The 
checkpoint diameter does not change between these levels. The total checkpoint diameter size reduction over 
5 minutes in condition B is therefore not known beforehand, and depends on the participants’ efficiency in 
mastering the task. 

After completing the helicopter control training, all participants receive the same exam, where they are 
required to fly one track with the smallest checkpoints used during the training. Trainee performance is 
determined by the number of checkpoints correctly flown through and the time needed to finish the track. A 
post-experiment questionnaire measured subjective ratings on the amount of challenge experienced. 

3.1 Apparatus 

3.1.1 BCI Tooling 

Neurosky Mindwave Mobile (see Figure 1 for a drawing of its components) is a single channel EEG device 
with a dry sensor positioned on the forehead (approximately Fpl position). The real-time processed measure 
used for BCI in this study is attention. Neurosky does not reveal the exact composition of this measure, but 
indicates that the attention is based primarily on beta waves. Attention is scaled from 1-100, with 
interpretations: 1-20 strongly lowered, 20-40 reduced, 40-60 neutral, 60-80 slightly elevated, 80-100 
elevated. 



Figure 2. Mindwave Mobile Figure 3. Screenshot of Helicopter Control Training Game 

3.1.2 Helicopter Control Training Game 

The Helicopter Control Training Game (see Fig. 2) is a low fidelity simulation environment developed using 
the Unity engine in the XLab at the Netherlands Aerospace Centre - NLR. The game is used to familiarise 
participants with basic principles of helicopter controls such as pitch, roll and yaw. The simulation features 
highly simplified helicopter flight models and controls, allowing for relatively easy mastery of basic flight 
control. The task is to fly through ‘augmented hoops’ in the sky. The hoops change from large to small in the 
time based condition, while in the mental state based condition the hoops vary as a function of attention level. 


4. RESULTS 

All participants completed the experiment successfully. Unexpectedly, participants in the BCI controlled 
condition did not perform better on the exam than participants in the time interval controlled condition (see 
Table 2 for results). 
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Table 2. Means and standard deviations (in brackets) of the results on the Helicopter Control Training Game for the 
conditions time based and mental state based control of task difficulty. Exam score indicates the average number of 
correctly flown checkpoints; experienced challenge indicates the average subjective rating from 1 to 10 

(1 = easy, 10 = hard) 


Condition 

Total sum deviation from 
optimal attention range 
during training 

Total time deviation 
from optimal 
attention range 
during training 
(seconds) 

Total 

training 

time 

(seconds) 

Exam 

score 

Experienced 

challenge 


839 

84 

335 

4.0 

7.25 

A Time based 

(323) 

(24) 

(13) 

(0.7) 

(1.3) 

B Mental state 

611 

58 

355 

2.0 

8.0 

based 

(255) 

(16) 

(34) 

(1.9) 

(1.4) 


Participants in the time based condition spent an average of 26 seconds more outside of the optimal 
attention range (25% of total training time) compared to participants in the BCI controlled condition (16% of 
total training time). For some participants, the attention level graphs showed clearly that whenever a 
participant’s attention level surpassed the threshold, the task difficulty would change, causing the 
participant’s attention level to normalize in turn. For other participants, BCI triggers are less clearly or not 
always associated to excess of the optimal attention range. For example, Figure 4 illustrates five correct BCI 
triggers, but the triggers at 135 and 210 seconds seem to be influenced by EEG spikes and lead to incorrect 
events. Later attention levels (at 250, 280 and 290 seconds) should have been detected and events should 
have been triggered. One participant (mental state based condition) remained in the optimal attention range, 
but kept performing poorly and ended up with zero correct checkpoints in the exam. 

Participants in the time based condition varied considerably in overall attention level (either very high or 
very low), but did not differ much in exam scores. For two participants in the time based condition the 
subjective ratings were inconsistent to the measured attention levels: intermediate challenging (5) versus high 
attention levels, and rather challenging (8) versus low attention levels. 


Participant 1 Attention level and triggers 
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Figure 4. Attention level and event triggers for participant 1 
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5. DISCUSSION 

This study was set up to determine whether BCI devices can be used to more effectively trigger scenario 
events in realistic training settings (as opposed to EEG laboratory conditions). The pilot study revealed the 
potential of BCI for training as well as some improvements to make. BCI using the Mindwave attention level 
functions reasonably well to adjust the task difficulty by increasing or decreasing the diameter of an 
augmented hoop in the sky. Some technical adjustments in the attention level criteria (such as dealing with 
EEG spikes) may increase reliability, while an adjusted size and timings of the hoops may improve the effect 
on the learning progress. Also, the allotted training time (five minutes) might not have been sufficient to 
significantly increase the performance of participants with poor initial skill level. For fair comparison of the 
conditions, the time intervals should be based on the average learning curve of the intended training 
audience. Ensuring sufficient statistical power of the test will require a larger sample of trainees who are 
more homogeneous with respect to game experience in general and experience with flight simulators in 
particular. 

The mindwave attention level may be used as a rough motivational indicator the trainees have to the task, 
but other EEG indicators may be more clearly linked to task difficulty (increased theta band) or cognitive 
load (reduced high alpha band). Using these measures will require some additional real time algorithms to be 
developed. BCI controlled training using EEG devices that are easy to apply in real training settings appears 
to be viable, although considerable effort is needed to ensure the measurements and the trigger events are 
well tuned to the training audience characteristics such as the learning curve. Based on the potential 
demonstrated in the current pilot experiment, the full experiment is intended to be performed after 
implementing the aforementioned improvements. 

Modern consumable EEG devices are promising in achieving adaptive training in real training 
organisations through maintaining optimal cognitive load for the trainee. This may enhance the training 
effectiveness of the training session while achieving personalized training trajectories. Fully automated 
training however will require improved modeling and measuring of learning and performance which can be 
very complex in professional settings. This effort is likely to pay off as the potential of reducing the average 
(formal) training time, personalize training schedules, and reducing trainee attrition has significant economic 
effects. 
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